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Item  20  (Continued) 

Approximately  30  pilots  participated  in  the  development  and  refinement  of  the  scenario.  Sixteen  pilots 
participated  in  the  actual  study.  The  subjects  were  experienced  fighter  pilots  transitioning  to  the  F-16  aircraft.  The 
objective  of  the  study  was  to  investigate  mission  planning  and  mission  effectiveness  as  a  function  of  experience  with 
the  simulation  scenario.  The  pilots  prepared  and  documented  two  attack  plans:  the  first  based  on  intelligence  briefing 
information  and  the  second  after  four  trails  in  the  ASPT/F- 16.  Thus,  each  pilot  flew  eight  trials  in  the  simulated  combat 
environment. 

The  results  indicated  improvement  in  weapons  delivery  and  survivability  skills.  An  analysis  of  mission  planning 
revealed  substantial  variability  amoung  the  pilots  in  their  attack  plans  and  little  correspondence  between  each  plan 
and  the  mission  as  flown.  The  tendency  to  execute  the  attack  as  planned  improved  somewhat  on  the  second  planned 
attack.  Pilot  opinion  indicated  a  need  to  enhance  the  scenario  by  adding  a  formation  attack  capability,  valid  threat 
performance  models,  larger  gaming  area  to  include  training  in  navigational  systems  operation,  and  additional  detail 
in  the  visual  scene  in  the  areas  of  low-level  flight.  The  addition  of  potential  enemy  aircraft  threats,  forward  air  controller 
in-flight  briefing  and  simulation  of  a  communications-jammed  environment  was  also  seen  as  valuable.  The  scenario 
as  constructed  was  viewed  as  an  acceptable  first-step  training  aid.  With  the  recommended  improvements,  most  pilots 
felt  that  the  training  potential  of  a  simulation  approach  to  combat  training  was  excellent,  particularly  in  the  areas 
of  skill  integration,  mission  planning,  and  threat  evasion  tactics  with  immediate  feedback. 
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SUMMARY 


Objective 

The  primary  objective  was  a  behavioral  and  engineering  baseline  for  a  simulated  surface-attack  mission  adapted 
to  the  deep-strike/interdiction  role  of  the  F-16  weapons  system. 


BackgrotwdfRatioaale 

Prior  training  research  and  development  (R&D)  efforts  had  demonstrated  positive  siraulator-to-aircraft  transfer  of 
training  for  F-5  and  A-10  air-to-surface  weapons  delivery  tasks  on  a  conventional  range.  Subsequently,  a  close-air- 
support  simulation  capability  was  developed  on  the  Advanced  Simulator  for  Pilot  Training  (ASPT)  for  application  to  the 
A-10  combat  role,  and  a  feasibility  experiment  was  conducted  in  which  mission-ready  A-10  pilots  flew  the  simulated 
combat  scenario.  The  results  demonstrated  increased  skill  in  both  offensive  and  defensive  categories  (target  kill  and 
survivability).  More  recently,  positive  transfer  of  training  from  a  simulated,  high  threat  combat  environment  to  the  Red 
Flag  exercise  was  demonstrated  for  the  A-10.  Based  on  these  results,  the  scope  of  the  R&D  was  extended  to  the  skills 
and  tasks  of  the  F-16  weapons  system. 


Approach 

The  ASPT  was  used  for  the  development  of  a  generic  deep-strike/interdiction  simulation.  A  scenario  was  developed 
specifically  for  the  F-16  weapons  systems  and  included  visual  target  penetration  and  attack  in  a  target  area  defended 
by  surface-to-air  missiles  and  anti-aircraft  artillery.  Sixteen  experienced  fighter  pilots  flew  the  scenario  according  to 
attach  plans  of  their  own  design.  Primary  data  included  mission  outcome  events,  planned  mission  ground  tracks,  and 
pilot  opinion. 


Specifies 

Method.  After  pre-mission  planning,  each  erf  the  F-16  pilots  was  given  four  trials  in  which  to  execute  that  plan  in 
the  ASPT.  Following  four  repetitions  of  the  initial  plan  (Plan  A),  each  pilot  left  the  simulator  in  order  to  prepare  a  second 
attack  plan  (Plan  B).  Each  pilot  then  executed  the  second  plan  four  times;  thus,  each  of  the  16  pilots  flew  a  total  of 
eight  passes.  The  scenario  included  target  area  visual  penetration  and  attack  in  an  environment  defended  by  two  surface- 
to-air  missiles  and  three  anti-aircraft  guns.  The  target  was  one  of  five  buildings  placed  along  a  road  located  in  a  valley 
bounded  on  the  east  and  west  by  mountainous  terrain.  Dependent  measures  included  (a)  automated  objective  data  of 
each  pass  in  terms  of  ingress,  egress,  weapons  release  status,  bomb  miss  distances,  number  and  type  of  threat  activity, 
time  within  threat  envelopes,  chaff  protection  time  and  number  of  releases,  and  overall  indications  of  altitude,  g-levels, 
and  airspeed,  (b)  video-tapes  and  experimenter-drawn  records  of  aircraft  ground  track,  and  (c)  pilot  responses  to  pre- 
and  post-mission  questionnaires.  Debriefings  were  also  conducted. 

Findings  and  Discussion.  Analyses  of  mission  outcome  events  showed  that  pilot  performance  improved  in  all 
categories.  Terminal  performance  levels  for  Plan  A  and  Plan  B  were  approximately  the  same.  No  pilot  executed  the  attack 
as  planned  on  the  first  set  of  trials.  No  two  pilots  planned  the  same  route.  The  majority  of  plans  took  advantage  of  direct 
terrain  masking  for  ingress  and  egress.  A  small  number  of  pilots  chose  to  attack  a  threat  site  as  well  as  the  target.  There 
was  a  greater  tendency  to  execute  the  attack  as  planned  on  the  second  set  of  four  passes. 


Conchitiowai'Roeomnsemdations 

The  simulation  of  tactical  scenarios  provides  an  opportunity  to  integrate  skills  in  mission  planning,  target 
acquisition,  weapons  delivery,  and  threat  evasion  under  simulated  combat  conditions. 
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The  existing  simulation  capabilities  of  the  ASPT  can  be  used  to  collect  data  that  define  requirements  for  generic 
scenario  simulation.  In  the  view  of  the  authors,  several  enhancements  would  have  beneficial  impacts  on  a  training 
scenario;  specifically,  (a)  more  low  altitude  cues  in  areas  of  likely  ingress  and  egress,  (b)  a  larger  gaming  area  to  provide 
a  greater  variety  of  attack  routes  and  more  systems-oriented  navigation  skills,  (c)  multi-ship  capability  for  practice  in 
planning  and  executing  formation  ingress,  attack,  and  rejoin  (light  coordination,  (d)  high  fidelity  modeling  of  ground 
threats  coupled  with  graphic  and  alphanumeric  representations  of  threat  versus  aircraft  engagements  to  enable  instruction 
in  defensive  maneuvering,  and  (e)  more  complete  avionics  and  navigation  systems  simulation.  All  of  these  enhancements 
are  within  existing  simulation  technology. 
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PREFACE 


This  effort  represent*  a  portion  of  the  research  and  development  (R&D)  program  of  the  Air  Force  Human 
Resource*  Laboratory  for  Technical  Planning  Objective  3,  the  thrust  of  which  ia  Air  Combat  Tactics  and 
Training.  The  general  objective  of  this  thrust  is  to  identify  and  demonstrate  cost-effective  training  strategies 
and  training  equipment  capabilities  for  use  in  developing  and  maintaining  the  combat  effectiveness  of  the  Air 
Force  aircrew  members.  More  specifically,  the  effort  was  part  of  the  R&D  program  conducted  under  the  Air 
Combat  Training  Research  subthrust,  which  ha*  a*  its  goal  to  provide  a  technology  base  for  training  high  level 
and  quickly  perishable  skills  in  simulated  combat  enviroment*.  Work  Unit  1 1230367,  F-16  Combat  Scenario 
Development  and  Evaluation,  addressed  a  portion  of  this  subthrust,  namely,  the  use  of  a  simulated  combat 
scenario  for  tactical  mission  planning  and  execution.  Mr.  James  F.  Smith  was  the  project  scientist  and  Mr. 
Robert  Woodruff  was  the  task  scientist.  Dr.  Elisabeth  L  Martin  and  Mr.  I.  Gavin  Lidderdale  were  coprincipal 
investigators. 

Mr.  Lidderdale  was  at  the  Operations  Training  Division  at  Williams  AFB  from  October  1979  to  April  1982 
as  the  Royal  Air  Force  participant  in  the  Exchange  Scientist  Program  sponsored  under  the  auspices  of  The 
Technical  Cooperation  Program  (TFCP).  Mr.  Lidderdale  has  since  returned  to  the  United  Kingdom  as  Head 
of  the  Human  Factors  Section,  Operational  Research  Branch,  Headquarters  Royal  Air  Force  Strike  Command, 
High  Wycombe,  Bucks,  England. 

The  authors  wish  to  express  their  appreciation  to  Ms.  Rebecca  Brooks  and  Dr.  Lowell  Schipper  whose 
assistance  in  data  collection  proved  invaluable.  We  also  appreciate  the  efforts  of  Maj  Richard  Engel  in  designing 
and  modifying  the  simulation  capability  available  on  the  ASPT  and  for  providing  inputs  to  the  experimental 
protocol.  Our  special  thanks  go  to  the  Air  Force  pilots  who  participated  in  the  developmental  phase  and  in  the 
conduct  of  the  actual  study. 
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SURF  ACE- ATTACK  MISSION  SIMULATION:  PRELIMINARY  SCENARIO  EVALUATION 


I.  INTRODUCTION 

For  a  pilot  to  achieve  combat  readiness,  several  categories  of  flying  skills  must  be  mastered  (Prophet  &  Monsees, 
1980):  (a)  airmanship  skills,  (b)  mission  skills,  and  (c)  combat  skills.  Basic  airmanship  skills  are  taught  in  undergraduate 
pilot  training  and  continue  to  be  reinforced  through  practice  in  all  flights.  Mission  skills  are  introduced  when  transitioning 
into  a  new  weapons  system  with  instruction  and  practice  continued  throughout  continuation  training.  Combat  skills 
represent  a  higher  task  level  that  involves  using  the  aircraft  as  a  tactical  weapon.  As  Prophet  and  Monsees  ( 1980),  pointed 
out  the  specific  tasks  and  skills  that  underlie  this  higher  level  are  not  well  understood.  Therefore,  designing  a  training 
environment  to  facilitate  and  maintain  skill  acquisition  is  a  difficult  task.  It  is  a  generally  accepted  notion  that  pilots 
need  to  train  under  the  same  conditions  as  those  under  which  they  will  be  expected  to  fight;  therefore  much  effort  is 
devoted  to  creating  combat-like  training  conditions. 

Readiness  training  for  ground  attack  mission  has  emphasized  the  need  for  pilot  participation  in  special  tactical  flight 
exercises.  These  special  exercises  are  an  attempt  to  augment  the  routine  training  regimen  with  more  realistic  exposure 
to  combat-like  conditions.  Typically,  these  special  exercises  are  held  on  a  recurrent  basis  with  each  location  having 
a  designated  name  and  purpose.  Some  of  the  better  known  ones  are  Red  Flag,  Maple  Leaf,  and  Cope  Thunder. 

Although  the  special  exercise  form  of  combat  simulation  has  many  positive  features,  several  restrictions  limit  the 
degree  of  “combat  fidelity.”  For  example,  the  minimum  altitudes  allowed  in  actual  flight  are  somewhat  higher  than  the 
altitudes  projected  for  use  in  many  modem  tactical  warfare  scenarios.  Further,  due  to  the  location  of  suitable  ranges 
for  staging  the  exercises,  there  are  severe  limitations  with  respect  to  exposure  to  various  types  of  terrain,  cultural  features, 
and  environment  conditions.  Due  to  financial,  logistical,  and  safety  limitations,  any  practice  delivering  live  ordnance 
of  some  types  is  also  limited  or  nonexistent. 

All  of  the  restrictions  in  the  previous  paragraph  have  the  effect  of  reducing  the  maximum  level  of  combat  fidelity 
that  can  be  attained.  To  the  extent  that  these  restrictive  practice  conditions  limit  the  extent  of  the  learning  experience, 
it  is  reasonable  to  assume  that  some  degree  of  potential  training  effectiveness  (i.e. ,  transfer  of  skills  to  actual  combat) 
is  lost.  In  addition,  other  restricitions  limit  the  degree  of  training  effectiveness  that  are  specifically  related  to  principles 
of  learning  and  training.  Perhaps  the  most  significant  limitation  is  the  infrequency  of  the  pilot’s  exposure  to  these 
programs.  (Depending  on  a  number  of  factors,  a  pilot  may  participate  in  a  special  combat  exercise  once  in  a  12-  to  20- 
month  period.)  This  is  not  often  enough  to  maintain  any  high  level  of  proficiency.  The  combat  training  syllabus  does 
not  always  provide  the  immediate  feedback  needed  to  facilitate  the  learning  process.  Finding  out  during  a  debriefing 
that  they  were  shot  down  3  hours  earlier  is  not  likely  to  receive  the  same  impact  as  being  informed  immediately  and 
required  to  leave  the  exercise.  It  is  difficult  for  the  pilots  to  even  remember  what  they  were  doing  and  thinking  during 
the  relevant  time  frame.  Too  often,  the  special  exercise  form  of  training  is  not  readily  adaptable  to  meeting  the  specific 
training  needs  of  the  individual  pilot  or  in  providing  all  participating  pilots  with  an  equal  opportunity  to  practice  the 
specific  tasks. 

Although  flight  programs  of  the  Red  Flag  type  are  considered  valuable,  all  of  the  previously  mentioned  restrictions 
limit  the  attainment  of  the  long  range  goal  of  true  combat  readiness.  Alternative  training  media  must  be  developed  and 
implemented  to  reach  a  combat-ready  goal  effectively.  Recent  developments  in  electronic  and  computer  technology 
appear  to  provide  a  capability  to  create  a  ground  attack  combat  training  environment  via  ground-based  flight  simulation. 

The  concept  of  using  synthetic  flight  devices  to  support  training  in  air  combat  training  is  n<  •  new ,  however  because 
of  limitations  in  visual  system  scene  complexity  levels,  the  primary  operatic,  d  application  ha.-  l>een  for  the  air-to-air 
mission.  Even  in  this  area,  limitations  in  visual  display  technology  h  v«*  precluded  a  fuff'  oeptable  tra  uug  apability. 
There  are  currently  no  operational  flight  simulators  that  can  fully  support  training  in  tactical  ground  attack  applications. 
The  extent  to  which  such  a  capability  could  provide  an  effective  training  adjunct  to  the  special  exercise  flight  programs 
is  now  being  addressed  with  an  exploratory  research  and  development  (R&D)  program  that  seeks  to  determine  the  training 
potential  and  the  simulation  requirements.  However,  the  real  issue  is  not  technology;  it  is  training.  Technology  can  only 
provide  the  medium.  The  issue  is  how  best  to  train  the  skills  and  tasks  required  for  mission  and  combat  level  effectiveness. 
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The  purpose  of  the  present  study  is  to  examine  the  |K)tential  of  extending  the  scope  of  mission  and  combat  readiness 
training  through  the  flight  simulation  medium. 


II.  BACKGROINI) 

Several  recent  Rdil)  efforts  have  demonstrated  the  effectiveness  of  simulator  training  in  the  area  of  conventional 
range  weapons  delivery  tasks  (Gray  &  f  uller,  1477;  Gray,  Chun.  Warner.  &  Fubanls,  1481 1.  This  training  concept  has 
also  been  extended  on  an  experimental  basis  to  the  tactical  ground  attack  domain.  A  feasibility  study  was  conducted 
to  demonstrate  the  potential  of  high  threat  tactical  ground  attack  simulation  scenarios  for  future  training  applications 
to  a  close  air  support  (CAS)  mission  (Kellogg.  Prather,  &  Castore.  1481 1.  In  this  study,  the  Advanced  Simulator  for  Pilot 
Training  (ASPT),  configured  as  an  A- 10  aircraft,  was  used  as  the  test  vehicle.  The  scenario  was  a  part-task  profile 
requiring  a  single  ship  target  area  penetration,  attack,  and  egress  under  visual  conditions.  A  tactical  gaming  area  of 
about  10  square  miles  was  modeled.  The  target  was  a  stationary  tank  defended  by  anti-aircraft  artillerv  I A  A  A)  and  surface- 
to-air  missiles  (SAMs).  Bomb  miss  distances  and  ground  threat  activity  were  recorded  in  realtime.  Thus,  feedback  to 
the  pilot  could  Ire  immediate  as  to  a  hit  or  miss  on  the  attack  and  the  success  of  threat  evasion. 

Seven  mission-ready  A-10  pilots  each  flew  20  attacks  into  the  threat/target  area.  The  following  results  were 
indicated: 

1.  The  second  10  trails  resulted  in  a  significant  increase  in  survival  and  offensive  outcomes  when  compared  to 
the  first  10. 

2.  The  first  10  trials  showed  marked  improvement  in  offensive  scores  but  not  in  defensive  scores. 

A.  The  last  10  trails  showed  an  increase  in  defensive  scores. 

The  subjective  opinions  of  the  pilots  were  favorable  towards  the  simulation  effort  particularly  because  it  provided 
immediate  feedback  in  an  interactive  threat  environment.  Immediate  and  reliable  information  is  tvpically  not  available 
during  an  in-flight  combat  exercise. 

More  recently,  positive  transfer  of  training  from  flight  simulator  to  Bed  flag  performance  has  been  reported  (Hughes. 
Brooks,  Graham,  Sheen,  &  Dickens.  1482).  This  experiment  used  the  ASPT/A-10  simulator  with  a  new  data  base 
modeled  after  a  real  tactical  engagement  range.  A- 10  pilots  received  2  hours  of  practice  in  simulated  GAS  and  battlefield 
interdiction  missions  in  a  simulation  of  an  electronic  warfare  env  ironment.  For  the  GAS  mission,  target  arrays  consisted 
of  three  groups  of  seven  tanks  each.  Targets  for  the  interdiction  missions  were  two  command  posts,  each  of  which  consisted 
of  a  group  of  four  vehicles.  Threats  were  simulated  AAA  guns,  and  SAMs.  All  threats  were  modeled  in  an  independent 
radar-controlled  mode.  The  results  showed  that  those  pilots  were  trained  in  the  simulator  survived  a  significantlv  higher 
proportion  of  total  tied  Flag  missions  than  did  a  control  group  of  pilots  who  did  not  receive  the  simulator  training. 

The  results  of  these  initial  experiments  were  encouraging  in  that  learning  effects  were  observed,  pilot  opinion  was 
generally  favorable,  and  positive  transfer  of  training  was  demonstrated.  Therefore,  a  deep-strike  scenario  tailored  for 
F-16  operations  was  designed,  using  the  original  data  base.  The  original  T-A7  configuration  of  a  second  ASPT  cockpit 
was  re-configured  to  an  F-16  cockpit.  The  target  was  changed  from  a  tank  to  a  storage  building,  and  the  threat  arrav 
and  performance  were  enhanced.  A  subject  protocol  involving  intelligence  briefing  and  mission  planning  was  developed. 

Several  preliminary  data  collection  efforts  were  conducted  in  this  environment  using  the  ASPT/F- 16  simulator  and 
experienced  fighter  pilots  undergoing  F-16  training;  approximately  AO  pilots  were  used  during  this  phase.  As  a  result 
of  their  performance  and  comments,  several  changes  were  made  to  the  simulation: 

1 .  The  visual  scene  models  of  the  surface-to-air  missiles  were  increased  in  size  to  conqiensute  for  a  lack  of  contrail 
or  smoke. 

2.  The  maximum  turning  g  capability  was  reduced  from  24g  to  I  fig  to  model  missile  (terformanre  capability  more 
realistically. 
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3.  The  threat  sites  were  programmed  to  deactivate  if  successfully  attacked. 

4.  A  previously  used  weather  ceiling  of  4500  feet  above  ground  level  I AGL.)  was  removed  because  the  ASFT/K-16 
radar  capability  was  not  used  in  this  study. 

5.  The  range  of  exercise  activation  was  increased  to  provide  a  sufficient  gaming  area  for  the  F-16. 

6.  The  initial  position  of  the  aircraft  was  changed  in  such  a  way  as  to  provide  a  neutral  starting  point. 

7.  The  pilot’s  choice  of  surface  attack  delivery  mode  and  the  type  of  ordnance  selected  were  recorded  and 
incorporated  into  the  performance  measurement  print-out. 

8.  A  subroutine  was  added  to  provide  some  defensive  countermeasures  capability  by  simulating  the  effects  of  chaff 
on  the  tracking  ability  of  the  missiles. 

The  phase  of  evt  .'ufiotiarv  development  was  stopped  when  no  new  significant  suggestions  arose  that  could  be  incorporated 
into  the  scenario  without  major  engineering  and  hardware  enhancements.  At  this  point  the  configuration  of  the  combat 
simulation  environment  was  "frozen"  (i.e.,  no  hardware  or  software  changes)  for  the  purposes  of  collecting  the  data  for 
the  present  study. 

In  the  preliminary  development  of  the  combat  scenario  used  in  this  study,  it  became  apparent  that  those  pilots  who 
took  the  time  to  carefully  study  the  intelligence  briefing  and  the  map  of  the  environment  were  less  likely  to  get  lost  and 
were  more  successful  in  finding  the  target  and  avoiding  the  threats  than  those  who  simply  glanced  at  the  materials  and 
“climbed  in."  For  the  latter  type  of  pilot,  much  time  was  spent  in  basic  area  orientation.  It  was  of  interest  to  examine 
this  effect  more  systematically.  One  skill/task  that  directly  contributes  to  effective  missions  in  mission  planning.  A  well- 
planned  mission  prepares  the  pilot  for  the  unexpected.  The  pilot  has  already  considered  contingencies  for  poor  weather, 
unbriefed  threat  activity,  and  communication  jammed  operations  and  has  alternatives  and  estimates  (even  if  subjective) 
of  probability  of  success  for  these  alternatives.  Proficiency  in  mission  planning  should  lead  to  skill  in  tactical  decision 
making  and  situation  awareness,  two  factors  thought  to  be  essential  for  an  effective  combat  pilot  (Prophet  &  Monsees, 
1980).  These  terms  are  vaguely  defined  at  best  and  most  often  recognizable  by  their  absence.  It  seemed  the  even 
rudimentary  preparation  enhanced  the  benefits  of  the  simulation  experience  and  prevented  many  errors  for  the  pilots 
who  participated  in  the  developmental  stage. 

The  simulation  scenario  provided  an  efficient  and  accessible  environment  in  which  a  pilot  could  “test"  an  idea  or 
strategy.  Repealed  opportunities  resulted  in  refinements  and  sometimes  completely  new  approaches.  It  was  of  interest 
to  study  how  pilots  would  change  their  strategy  once  they  had  acquired  first-hand  experience  in  the  environment. 
Thorndvke  (19801  postulated  that  the  spatial  knowledge  derived  from  map  study  is  quite  different  than  the  knowlcdgc 
gained  from  actual  navigational  experience.  He  labels  the  type  of  knowledge  gained  from  map  study  as  survey  know  ledge 
characterized  by  knowledge  of  locations  and  F.uclidean  distances  between  objects.  Ke  argues  that  spatial  reasoning  tasks 
require  a  different  type  of  knowledge  which  could  come  from  a  combination  of  survey  and  navigational  knowledge. 
Presumably,  effective  mission  planning  involves  a  good  deal  of  spatial  reasoning.  Therefore,  a  pilot  would  lienefit  from 
(with  kinds  of  experience.  A  protocol  developed  for  this  study  would  require  plannng  from  map  study  and  subsequent 
planning  based  upon  navigation  experience.  Although  it  would  be  desirable  to  evaluate  the  goodness  of  the  two  types 
of  plans,  evaluation  is  not  a  simple  matter.  As  Hayes-Roth  (1980)  pointed  out,  evaluation  of  a  plan  is  difficult  due  to 
inherent  ambiguity.  The  use  of  multiple  criteria  is  possible,  but  the  weighting  of  the  criteria  needs  to  lie  empirically 
determined.  One  approach  is  to  evaluate  the  performance  of  the  plan.  That  approach  is  implicitly  consistent  with  the 
approach  taken  in  the  present  study.  However,  it  must  be  recognized  that  the  soundness  of  tile  plan  and  the  pilot's  abilitv 
to  execute  the  plan  are  two  distinct  entities.  In  the  present  study,  there  is  no  attempt  to  evaluate  the  plan  independently 
from  the  execution  of  the  plan. 

The  scenario  was  constructed  to  represent  a  single-ship  daytime  visual  target  area  penetration  to  a  storage  building 
target.  The  target  was  located  in  a  valley  bounded  on  the  east  and  west  by  mountainous  terrain.  The  immediate  target 
vicinity  was  defended  by  two  SAMs  and  three  AAA  emplacements. 


The  general  procedure  for  data  collection  Involved  each  pilot's  developing  a  plan  for  attack  based  on  information 
made  available  in  a  written  intelligence  briefing.  The  pilot  then  proceeded  to  execute  that  plan  in  the  ASPT  scenario. 
Following  the  initial  exposure  to  the  environment,  each  pilot  then  formulated  a  second  attack  profile  and  was  again  given 
the  opportunity  ’  >  execute  that  plan  in  the  ASPT. 

To  summarize,  the  study  presented  in  this  report  represents  an  extension  of  the  earlier  work  in  a  simulated  hostile 
environment.  The  objective  of  this  study  was  to  establish  a  behavioral  data  base  of  pilot  performance  in  a  simulated  deep- 
strike/interdiction  surface  attack  scenario.  The  data  base  will  be  used  to  define  the  requirements  of  simulated 
environments  in  support  of  tactical  surface  attack  training.  The  scenario  incorporated  elements  of  the  following  tasks 
and  skills:  (a)  mission  planning,  (b)  target  area  situation  awareness,  (cl  target  acquisition,  (dj  weapons  delivery,  and 
(el  ground  defensive  evasion.  Tile  research  issues  were  to  define  tasks  and  skills  that  can  be  effectively  practiced, 
performance  measurement  requirements,  ami  simulation  requirements  tosupport  the  training  objectives. 


111.  METHOD 


Equipment 

The  ASPT  used  in  the  conduct  of  this  study  is  a  research  simulator  originally  designed  with  a  full  mission  T-37 
capability.  A  detailed  description  of  the  original  device  may  be  found  in  Gum,  Albery,  and  Basinger,  1975.  One  cockpit 
has  been  modified  to  an  A- 10  configuration,  while  the  other  cockpit  has  been  configured  as  an  F  -16  aircraft.  Both  systems 
were  designed  to  have  necessary  cockpit  and  aerodynamic  capabilities  to  support  transition  flight  tasks,  such  as  takeoffs, 
approaches  and  landings,  instrument  flight,  basic  navigation  tasks,  and  conventional  air-to-ground  weapon  delivery. 
Neither  of  the  modified  configurations  has  full  mission  capabilities. 

The  following  is  a  description  of  the  ASPT/F  - 16  as  it  was  configured  during  the  conduct  of  the  present  study.  The 
visual  display  was  a  monochromatic  computer-generated  image  displayed  through  seven  CRTs  with  a  ±150°  horizontal 
by  +  1 10°  vertical  field  of  view.  (The  aircraft  field  of  view  is  360°  horizontal  by  + 180°  vertical  -40°  over  the  side  and 
—  15°  over  the  nose.  I  The  ASPT  also  has  —  15°  over  the  nose,  but  —37°  over  the  left  side  and  —  15°  over  the  right  side. 
The  A5PT/F- 16  does  not  have  the  capability  to  use  the  platform  motion  simulation  system  available  on  the  original  T-37 
configuration.  No  kinesthetic  device  g-cueing  was  used  in  the  present  study.  The  cockpit  layout  was  designed  to  duplicate 
the  Block  I  aircraft  in  most  major  tespects,  except  for  the  seat  which  was  a  modified  T-37  seat  titled  back  27°. 

Instrumentation.  All  flight  and  engine  instruments  were  operable  with  the  exception  of  the  fuel  flow  gauge.  The 
horizontal  stability  indicator  was  operable  but  not  the  inertial  navigation  system.  Communications,  chi  T/flare,  and 
electronic  countermeasure  panels  were  static  mock-ups.  The  head-up  display  (HUD)  was  an  operational  m  del  driven 
by  a  simulated  flight  control  computer. 

Aerodynamics  Model.  Aircraft  aerodynamics  were  modeled  from  sea  level  to  40,000  feet  from  0  to  0.9  mach.  and 
to  a  maximum  of  30°  angle  of  attack.  The  simulation  will  allow  higher  and  faster  flight  but  without  proper  atmospheric 
modeling  and  drag  coefficients.  Flngine  performance  was  modeled  from  idle  to  after  burner  and  from  sea  level  to  55,000 
feet. 


Weapons  Systems.  The  configuration  included  air-to-ground  simulation  of  manual  (bomb  and  strafe),  continuously 
computed  impact  point  (CCIP)  (bomb  and  strafe),  and  dive  toss  deliveries.  The  potential  ordnance  included  BDU  33, 
CBU  68,  MK  61,  MK  82  (high  and  low  drag),  and  MK  106.  For  those  modes  and  ordnance,  the  Stores  Management 
System  displayed  proper  indications  for  the  bombing  and  strafe  modes,  as  well  as  the  current  aircraft  inventory.  The 
simulation  did  not  include  air-to-ground  missile  deliveries  or  air-to-air  capabilities.  The  aerodvnamic  model  did  not 
account  for  weapon  weight. 

In  addition  to  the  flight  simulation,  the  ASFTVF-16  provided  several  advanced  instructional  features  that  were  used 
in  this  study.  A  camera  display  of  the  HUD  was  projected  at  the  advanced  instructor  console.  Graphic  displays  of  the 
terrain  features  and  aircraft  position  were  also  available  at  the  console;  either  of  these  could  be  videotaped  with  associated 
voice  communications.  The  current  status  of  the  aircraft  and  ground  defense  was  displayed  on  an  alphanumeric  CRT. 
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Environment  Description 


The  general  environment  (Figure  1 1  consists  of  a  broad,  flat  plain  with  mountains  surrouding  the  target  rone.  located 
on  the  eastern  edge  of  the  environment  are  two  parallel  mountain  ridges  separated  by  a  valley  which  contains  numerous 
high  jutting  pinnacles.  located  on  the  western  edge  of  the  environment  is  a  low  mountain  plain  approximately  700  feel 
about  the  surface.  Adjacent  to  this  low  mountain  plain  to  the  east  is  a  series  of  three  medium  altitude  mountains  mo 
higher  than  2100  feet),  located  in  the  southern  portion  of  the  environment  are  two  large  mountains,  6500  feet  at  the 
highest  point,  with  a  saddleback  between  them.  In  the  southeast  comer  of  the  environment  are  two  moderate  size 
mountains  approximately  2250  feet  high.  In  the  immediate  target  vicinity  (approximately  1.5  milesl  there  was  a  series 
of  35-foot-high,  inverted  tetrahedrons  (cone-shaped).  These  objects  have  black  sides  with  white  triangular  tops  and  are 
regularly  spaced  at  1000-foot  intervals.  The  purpose  of  these  cues  was  to  provide  depth  and  movement  information  to 
the  pilots. 

The  target  was  a  storage  building  located  on  the  south  side  of  a  road  that  runs  east-west  between  the  small  mountain 
ridge  to  the  southeast  and  the  two  large  mountains  located  in  the  southern  portion  of  the  environment.  The  target  was 
the  second  building  (of  five)  west  of  center.  These  buildings  were  located  alongside  a  road  and  approximately  halfwav 
between  the  large  mountain  and  the  intermediate  mountain  range  (Figure  2t.  The  buildings  were  approximately  .50  feet 
long,  25  feet  wide,  and  20  feet  high.  The  buildings  were  shaded  white. 


Threat  Simulation 

SAMi.  There  were  three  predominant  threat  sources  within  the  environment.  Two  of  the  sources  consisted  of  a  >AM 
and  four  quad-mounted  23mm  AAA  guns.  A  third  gun  site  was  located  in  the  immediate  vicinity  of  the  target  The  SAMs 
have  a  line-of-sight  capability  to  approximately  40,000  feet  with  a  2-1/2°  look  angle  and  minimum  altitude  of  .50  feel 
The  missiles  operated  as  radar  guided  only  (w  ith  a  maximum  (5g  turning  capability  I  and  required  It)  seconds  from  .intial 
acquisition  to  launch.  Direct  terrain  masking  was  effective  against  the  missile  radar.  The  operating  logic  was  indejieiidefit 
for  the  two  SAMs  but  the  first  SAM  to  acquire  the  aircraft  had  priority  for  visual  display.  Only  one  of  the  two  missiles 
could  be  launched  and  lie  in  flight  at  one  time  (due  to  restriction  in  the  number  of  moving  models  that  can  Is*  visually 
displayed).  After  the  missile  in  flight  aborts,  another  missile  from  the  same  site  or  a  missile  from  the  other  site  could 
then  proceed  with  the  launch  cycle.  The  missiles  were  visually  modeled  approximately  100  feel  in  length  and  shaded 
white.  No  smoke  at  launch  or  in-flight  signatures  were  modeled.  A  SAM  kill  was  defined  by  a  miss  distance  from  the 
aircraft  of  200  feet  or  less. 

,4A4.  The  anti-aircraft  guns  were  modeled  to  Ik*  visually  controlled  by  proficient  (nine-level)  gunners.  The  range 
of  the  guns  was  approximately  3000  feet.  The  program  controlling  the  guns  recorded  a  kill  if  the  aircraft  was  within  their 
envelope  for  4  consecutive  seconds.  Jinking  at  greater  than  4g  would  defeat  the  AAA  by  resetting  the  4-second  counter 
(Figure  3).  The  gun  sites  were  modeled  as  black  tubes  with  strobe  flashes  (to  simulate  firing)  at  the  base.  The  gun  site 
defending  the  target  was  not  visually  modeled  Two  visually  modeled  but  inactive  gun  sites  were  located  in  the 
environment  (one  was  hx-ated  in  a  valley  on  the  western  portion  lietween  the  plains  and  three  (waked  ridges;  the  other 
was  located  in  the  mid-vallev  north  of  the  active  northern  threat  site). 


Rules  of  Engagement 

The  pilot’s  overall  objective  wso  to  enter  the  target  area  from  the  start  (Hisition  indicated;  acquire,  engage,  and 
destory  the  target;  and  egress  the  target  area.  The  target  area  was  defined  as  a  4-mile  circle  around  the  target.  If  the 
aircraft  entered  the  target  area  and  subsequently  left  the  4-mile  area,  the  exercise  automatically  terminated  and  was 
scored  as  a  safe  exit.  There  were  three  means  by  which  the  aircraft  could  Iw  destroyed,  (a)  SAM  kill,  (bl  A  A  A  kill,  ami 
(e)  terrain  crash.  F.ach  trial  in  the  environment  was  independent  of  the  previous  trial.  That  is,  all  ordnance  was  restored 
and  the  threats  reactivated  for  the  next  trial  even  if  all  bombs  were  used  and  all  target  or  threats  destroyed  on  the  previous 
trial.  The  target  and  threat  sites  were  considered  destroyed  if  a  bomb  miss  distance  of  less  than  30  meters  was  obtained. 
The  stores  management  set  was  programmed  with  six  BDU-33  bombs  dropped  in  pairs  using  the  (’.('.IP  delivery  mode. 
Three  bomb  drops  were  allowed  (but  not  required)  on  any  trial.  Although  the  target  was  the  primary  objective,  the  threat 
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sites  could  be  destroyed.  If  a  threat  was  destroyed,  both  the  AAA  and  SAM  at  that  site  were  deactivated  for  the  remainder 
of  that  trial  only.  When  destroyed,  the  SAM  missile  disappeared  from  the  launch  pad.  The  strobe  flashes  from  the  AAA 
continued  but  no  kills  could  be  achieved. 

Missile  Warning.  An  auditory  tone  delivered  through  the  headsets  was  provided  as  a  cue  when  the  SA  M  was  tracking 
the  aircraft.  The  frequency  of  beeps  increased  as  the  missile  proceeded  through  the  acquisition  to  launch  phases.  (There 
was  no  cockpit  instrumentation  for  radar  hazard  warning.)  Acquisition  cues  were  provided  for  7  seconds,  track  for  3 
seconds,  and  throughout  missile  flight.  There  was  no  warning  to  the  pilot  who  was  being  tracked  by  the  A  AAs 

Chaff.  Chaff  was  simulated  and  had  to  be  dispensed  manually  by  the  pilot  (i.e.,  no  programmed  release  patterns); 
3  to  8  seconds  of  protection  was  afforded  by  each  bundle  (actual  value  is  randomly  selected).  The  chaff  protection 
simulation  logic  denied  the  SAM  missile  updated  aircraft  position  necessary  for  guidance  for  the  time  interval  of 
protection.  During  this  period  of  time,  the  SAM  continues  to  project  based  on  last  know  aircraft  position. 

Bomb  Simulation.  A  bomb  fragmentation  envelope  was  simulated  based  on  published  Mark  82  data.  When  a 
weapon  was  released,  a  timer  was  started  based  on  the  time  to  impact.  After  1,3,5,  7,  and  9  seconds  past  bomb  impact, 
aircraft  position  was  compared  to  impact  point.  If  the  radial  distance  was  less  than  that  of  the  tabled  data,  a  “frag”  damage 
flag  was  set  true  to  the  advanced  operator  console.  This  did  no  affect  the  aircraft  performance.  The  information  was 
provided  for  pilot  information  only. 

If,  during  the  exercise,  the  aircraft  was  destroyed  by  either  SAM  or  AAA,  it  was  programmed  to  begin  a  slow  spiral 
to  the  earth  with  all  control  inputs  nonfunctional.  Upon  terrain  crash,  the  visual  display  turned  uniformly  grey,  and  the 
aircraft  was  respositioned  at  the  starting  point  for  the  next  trial . 


Protocol 

The  pilots  received  the  equivalent  of  four  missions  of  transition  and  weapons  delivery  instruction  in  the  ASPT/F-16 
prior  to  participation  in  this  study.  Two  of  the  subjects  in  this  study  were  F-16qualified  instructor  pilots. 

On  completion  of  the  ASPT  traning  syllabus,  the  pilots  were  given  a  written  and  verbal  briefing  detailing  the  scenario 
and  purpose  of  the  study.  Written  materials  included  an  intelligence  briefing,  a  small  computer-drawn  map  of  the 
environment,  a  Mark  82  frag  envelope  data  graph,  a  large  hand-drawn  map  of  the  environment.  The  verbal  briefing 
explained  the  purpose  of  the  study,  summarized  the  contents  of  an  intelligence  brief,  and  outlined  the  protocol  that  the 
pilots  would  be  expected  to  follow.  It  was  emphasized  that  the  data  for  each  subject  would  be  kept  confidential  and  would 
be  coded  by  an  ID  number  only.  It  was  also  empahsized  that  there  would  be  no  attempt  to  evaluate  the  individual  pilot 
on  tactics  or  (lying  ability. 

The  pilots  were  asked  to  develop  an  attack  plan  based  on  the  information  available  to  them.  They  were  told  that 
they  would  subsequently  be  asked  to  execute  their  plan  four  times  in  the  ASPT.  The  pilots  were  instructed  to  draw  their 
planned  ground  track  on  the  large  map  and  to  complete  two  questionnaires  regarding  various  aspects  of  their  plan.  They 
were  also  asked  to  develop  their  plan  individually  and  not  to  discuss  it  with  other  pilots.  The  pilots  were  provided  with 
working  space  and  the  necessary  materials  (ruler,  compass,  protractor,  etc.)  for  individual  mission  planning.  When  it 
was  necessary  to  place  more  than  one  pilot  in  an  office,  every  attempt  was  made  to  prevent  collaborative  planning  by 
providing  close  supervision. 


Following  the  completion  of  the  planning  effort,  each  pilot  was  asked  to  fly  four  repetitions  of  his  plan  in  the  ASPT. 
Pilots  were  not  permitted  to  observe  the  performance  of  others.  Following  each  pass,  the  pilot  was  given  verbal  feedback 
as  to  the  outcome  of  the  trial  (bomb  miss  distance,  threat  activity,  source  of  termination,  frag  damage,  etc. ). 

Following  the  completion  of  the  fourth  pass,  the  pilot  was  given  two  postmission  questionnaires,  another  set  of  maps, 
and  computer  printouts  of  each  pass.  (Format  of  the  printouts  is  described  in  the  performance  measurement  section.  I 
The  pilots  were  asked  to  formulate  a  second  plan  in  the  same  manner  as  the  first  plan  and  told  that  they  would  be  asked 
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to  execute  four  repetitions  of  the  second  plan.  The  same  protocol  (i.e.,  the  planning  exercise  and  experimental  trials) 
was  followed  on  the  second  plan  as  had  been  followed  on  the  first  plan. 


Whenever  possible,  a  group  debriefing  was  held  to  provide  an  opportunity  for  discussion  of  problems  and  potential 
areas  of  improvement  to  the  scenario  and  simulation  capabilities.  When  it  was  not  possible  to  meet  as  a  group,  an 
individual  debriefing  was  conducted.  The  debriefings  were  not  formal  or  structured  in  any  systematic  fashion,  however, 
the  comments  were  noted  and  are  discussed  briefly  in  the  results  section  of  this  report. 


Subjects 

Sixteen  Air  Force  pilots  participated  in  this  study.  Two  of  these  were  F-16  instructor  pilots.  The  remaining  pilots 
were  all  previously  experienced  fighter  pilots  transitioning  into  the  F-16.  Their  time  in  fighter  aircraft  is  as  follows:  x 
=  1465  hours  (SD  =  750),  minimum  =  500,  maximum  —  2850.  Their  time  in  other  operational  aircraft  (not  including 
UPT  or  non-military)  is  as  follows:  x  =  907  hours  (SD  =  773),  minimum  =0,  maximum  =  1725.  Their  ages  ranged 
from  30  to  42  years  with  an  average  age  of  36  (SD  =3). 


Performance  Measurement 

Pilot  performance  in  the  high  threat  scenario  was  monitored  and  recorded  in  three  ways:  (a)  automated  objective 
measures,  (b)  videotape  recording  of  a  graphic  display  depicting  the  environment  and  the  movement  of  the  aircraft  in 
the  environment  with  associated  audio  track,  and  (c)  hand-drawn  tracings  made  in  real  time  of  the  aircraft  in  the 
environment  with  annotations  of  total  time,  significant  mission  events,  and  computer  malfunctions. 

The  videotapes  and  hand-drawn  tracings  were  used  as  complementary  information  to  that  obtained  from  the 
automated  performance  measures  and  were  used  to  assess  the  correspondence  between  the  pilot's  planned  mission  and 
the  actual  performance. 

The  development  of  the  automated  measurement  capability  used  in  this  study  took  place  over  a  period  of  3  months 
prior  to  data  collection  and  represents  a  significant  enhancement  to  the  research  and  training  tools  available  in  the  initial 
A- 10  feasibility  study.  The  data  were  collected  using  the  ASPT  data  record  capability  with  start/stop  and  computation 
logics  controlled  through  preprogrammed  algorithms;  the  sampling  rate  was  30  Hz.  Performance  measures  were  available 
shortly  after  the  termination  of  the  trial  on  a  line-printer  hard  copy  only  (no  data  storge  or  retrieval  system).  The  printout 
is  arranged  according  to  pilot  and  trial  identification  information,  aircraft  position  captures,  target  data,  threat  data,  and 
termination  data. 

The  following  data  were  available  from  the  automated  measurement  routine:  (a)  subject  identification  number,  (bl 
pass  number,  (c)  weapon  type,  (d)  weapon  mode,  (e)  aircraft  parameter  capture  values  at  the  points  of  ingress,  weapons 
release,  and  egress  (altitude,  airspeed,  bank  angle,  heading,  distance,  and  azimuth  to  target  and  probable  target),  (0 
time  elapsed  between  ingress,  weapons  release,  and  termination,  (g)  average  heading  and  altitude  between  those  events, 
(h)  maximum  and  minimum  altitude  between  those  events,  (i)  total  time,  (j)  bomb  miss  distances  in  feet,  (k)  overall 
maximum,  minimum,  and  average  altitude,  (1)  maximum  g-force,  (m)  source  of  termination,  (n)  number  of  rounds  fired 
by  each  threat,  (o)  percentage  of  time  aircraft  was  within  SAM  and  AAA  tracking  envelopes,  (p)  whether  a  threat  site 
was  destroyed,  (q)  number  of  chaff  release,  (r)  time  protected  by  chaff,  (s)  SAM  and  AAA  site  active  at  termination, 
(t)  SAM  kill  occurred,  time  of  missile  flight  and  miss  distance  of  missile,  (u)  tabular  and  numeric  representation  of  bomb 
distance  relative  to  target  and  clock  position,  (v)  if  aircraft  entered  its  own  frag  envelope,  the  envelope  time  and  distance, 
and  ( w)  maximum  angle  of  attack . 
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IV.  RESULTS 


The  results  of  the  study  are  discussed  under  the  headings  of  mission  planning,  mission  outcomes,  and  pilot  opinion. 
It  should  be  emphasized  that  the  pilots  who  participated  in  this  study  were  not  yet  F-16  qualified  (with  the  exception 
of  the  two  instructor  pilots),  had  not  yet  received  ground  training  in  F-16  tactical  surface  attack  operations  and  tactics, 
and  did  not  receive  any  instruction  during  the  conduct  of  the  study.  In  addition,  limitations  in  the  simulation  capability 
and  rules  of  engagement  precluded  exploiting  the  F-16  to  its  maximum  potential.  Therefore,  the  data  presented  in  this 
section  should  not  be  considered  representative  of  operational  mission-ready  capability. 


Miaaion  Planning 

The  problem  offered  to  the  pilots  was  to  plan  a  flight  path  that  minimized  exposure  to  threats  and  maximized  the 
opportunity  to  bomb  the  target.  For  their  planning  activities,  the  pilots  were  given  an  intelligence  briefing,  a  simplified 
map  of  the  target  gaming  area,  rulers,  protractors,  and  as  much  time  as  they  needed.  The  time  required  to  plan  the  initial 
mission  was  approximately  1  to  1.5  hours.  The  planning  activity  for  the  second  phase  required  slightly  less  time  (.75 
to  1.0  hour). 

The  analysis  of  the  mission  plans  shows  at  least  three  itemB  of  interest  : 

1 .  The  degree  of  agreement  between  the  pilots'  plans. 

2.  The  changes  in  the  content  of  the  plans  as  a  function  of  familiarity  and  experience  in  the  environment. 

3.  The  extent  to  which  the  planned  missions  corresponded  to  those  actually  flown. 

The  data  sources  available  to  address  these  three  items  included  the  flight  plans  as  depicted  by  the  pilots  on  their 
maps,  the  flight  path  tracings  made  during  the  experimental  trials,  the  automated  performance  measures,  and  the  pilot's 
responses  to  the  premission  and  postmission  planning  questionnaries.  The  primary  data,  though,  were  the  ground  tracks 
as  depicted  in  the  pilot-drawn  plans  and  experimenter-drawn  tracings.  Each  of  these  was  coded  by  reference  to 
geographical  position  relative  to  the  major  terrain  features  (i.e. ,  mountains,  roads,  hills,  etc.).  In  order  to  compare  they 
planned  route  with  the  routes  actually  flown,  the  sequences  (planned  vs.  flown)  of  terrain  features  were  compared.  A 
deviation  from  the  plan  was  scored  for  each  discrepancy  between  the  sequence  of  terrain  features  as  coded  for  the  plan 
versus  the  sequence  coded  for  the  pass  as  flown.  If  a  pass  was  terminated  due  to  a  threat  kill  or  terrain  crash,  only  the 
sequence  flown  to  that  point  was  considered.  Therefore,  if  a  pilot  was  shot  down  by  a  missile  but  had  flown  the  ground 
track  to  that  point  as  planned,  no  deviations  were  scored. 

Several  shortcomings  of  this  analysis  are  apparent.  The  analysis  was  confined  to  ground  track  and  did  not  include 
airspeed  or  altitide  data.  Thus,  for  example,  it  was  possible  for  a  pilot  to  make  large  deviations  in  altitude  from  that 
indicated  in  the  plan  without  their  being  scored  as  deviations.  In  addition,  this  analysis  does  not  take  into  account  the 
tactical  signficance  of  the  observed  deviations.  It  is  quite  possible  that  some  deviations  had  major  tactical  implications 
(e.g. ,  earlier  exposure  to  a  threat)  while  others  did  not.  Based  on  this  analysis  procedure,  the  following  questions  were 
addressed. 

1.  What  degree  of  similiarity  occurred  in  the  plans?  The  answer  to  this  question  is  simple.  No  two  pilots  planned 
th<-  same  attack  route.  This  diversity  is  surprising  in  lights  of  the  relative  simplicity  of  the  environment  and  scenario. 

2.  How  did  the  plans  change  as  a  function  of  familiarity  and  experience  in  the  environment?  Four  pilots  repeated 
the  same  plan  (i.e.,  no  change)  from  the  first  four  trials  to  the  serond  four.  Six  pilots  simply  reversed  the  sequence  of 
their  route.  The  remaining  six  pilots  made  changes  that  could  be  characterized  as  "new.”  One  of  these  pilots  constructed 
four  individual  plans  (one  for  each  pass).  It  is,  thus,  difficult  to  summarize  the  influence  of  familiarity  and  experience. 
Many  factors  (other  than  success  or  failure)  could  act  as  an  impetus  for  making  (or  not  making)  significant  alterations 
in  the  plans.  Considering  responses  made  on  the  questionnaires  and  during  debriefings,  it  is  clear  that  no  simple 
performance  rules  (e.g.,  win,  stay-lose,  shift )  will  account  for  choices  and  decisions  made  on  the  plans.  For  example. 


of  the  four  pilots  who  chose  to  repeat  their  initial  plan,  (a)  two  indicated  that  they  did  so  because  they  had  not  achieved 
the  success  they  felt  they  should  have  (one  in  fact  had  been  shot  down  on  three  of  the  four  passes},  (b)  one  had  been 
successful  in  the  first  plan  and  saw  no  reason  to  change,  and  (c)  one  thought  it  was  the  most  tactically  reasonable 
alternative.  Several  pilots  indicated  their  plan  B  was  an  option  that  they  had  originally  considered  but  disregarded,  and 
they  wanted  to  try  it.  Several  pilots  felt  their  original  plan  was  inadequate  because  some  aspects  of  the  environment  were 
significantly  different  from  what  they  expected  (e.g. ,  spatial-distance  relationship,  difficulties  in  tow-level  flight). 

3.  Were  the  missions  flown  as  planned?  If  the  ground  tracks  as  flown  contained  any  deviations  from  the  ground 
tracks  depicted  on  the  planning  maps,  the  mission  was  considered  to  be  different  from  the  plan.  The  results  of  this  analysis 
indicated  that  no  pilot  flew  all  four  passes  of  Plan  A  as  planned.  Of  the  total  68  Plan  A  passes,  only  three  (two  by  the 
same  pilot)  were  flown  as  planned.  During  the  second  set  of  missions,  there  was  a  greater  tendency  for  the  pilots  to  fly 
their  missions  as  planned.  Three  pilots  flew  each  of  the  four  passes  without  deviation.  Of  the  entire  set  of  Plan  B  passes, 
20  were  flown  as  planned.  Thus,  it  appears  that,  while  it  is  unlikely  a  mission  will  be  flown  as  the  pilot  planned  it, 
there  is  a  greater  correspondence  between  the  plan  and  the  actual  flight  as  a  function  of  experience  and  familiarity  with 
the  environment. 


Tactical  Consideration 

No  formal  attempt  was  made  to  assess  the  effects  of  different  tactics  in  th.s  oenario;  however  several  descriptive 
statements  can  be  made  about  the  way  the  pilots  chose  to  accomplish  the  mission.  In  general,  the  passes  were 
characterized  by  high  speed  (450  to  540  knots),  terrain  masked,  low  to  medium  altitude  (less  than  500  feet  ACL)  ingress 
and  egress  with  a  single  pass  on  the  target. 

Ingress.  All  subjects  except  one  took  advantage  of  terrain  masking.  (One  pilot  planned  two  of  his  Plan  B  passes 
using  a  head-on  ingress  route,  which  could  be  accomplished  without  detection  by  maintaining  an  altitude  profile  below 
the  2.5  degree  look-angle  of  the  missile. )  There  was  some  measure  of  agreement  for  those  who  chose  an  easterly  ingress. 
These  were  generally  planned  and  flown  at  a  low  to  medium  altitude  (100-500  feet  AGL),  which  was  lower  than  required 
by  the  masking  terrain's  elevation.  Westerly  ingresses  were  generally  flown  over  the  top  of  the  far  western  flat  plateu 
(elevation  700  feet)  but  below  the  ridge  lines  of  the  inner  three  mountains  of  the  valley.  Although  peak  elevations  ranged 
from  1450  to  2100  feet,  the  ridge  lines  were  of  graduated  elevations;  therefore,  this  route  also  required  a  medium  AGL 
flight  path  in  order  to  avoid  detection. 

Attack.  An  easterly  ingress  was  generally  associated  with  a  pop-up  attack  pattern.  Planned  pop-ups  were  often 
replaced  on  later  repetitions  by  low  angle  “non-standard  deliveries.”  A  westerly  ingress  over  the  saddle-back  resulted 
in  a  30  to  40  degree  dive  bomb  delivery.  If  a  pilot  chose  to  attack  more  than  one  of  the  targets  (e.g. ,  the  target  building 
followed  by  a  threat  site),  low-level  deliveries  or  shallow  pop-ups  were  used.  Several  pilots  executed  multiple  releases 
(accomplished  manually)  on  the  target.  Subsequent  attacks  on  the  same  target  were  not  planned,  but  several  were 
attempted. 

Egress.  Most  pilots  planned  a  low-level  high-speed  recovery  accompanied  by  horizontal  jinking  toward  the  nearest 
positions  with  terrain  masking  potential.  Afterbunner  was  frequently  used  during  egress.  Evasive  manuevering  from 
SAMs  in  flight  occasionally  forced  the  pilot  into  a  position  unmasked  by  terrain.  In  the  latter  cases,  the  trial  would  result 
in  a  safe  egress  if  the  pilot  arrived  at  the  preprogrammed  automatic  termination  position  even  if  a  SAM  with  a  lethal 
flight  path  was  in  flight.  This  situation  occurred  on  several  trials  and  safe  egresses  were  recorded  when  in  fact  given 
a  little  more  flight  time,  a  SAM  kill  would  have  occurred. 
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Miarfo*  Effective 


In  the  ideal  case,  a  totally  successful  mission  would  be  in  the  natural  product  of  a  carefully  constructed  and  well 
executed  mission  plan.  However,  as  noted  in  the  mission  planning  analysis  section,  there  was  considerable  variety  in 
the  plans  and  little  correspondence  between  the  plana  and  the  observed  performance,  thus  making  it  impossible  to  analyze 
effectiveness  from  the  perspective.  For  the  purpose  of  this  study,  mission  effectiveness  is  defined  in  terms  of  outcome 
events,  considered  independently  from  a  pilot's  plan  or  correspondence  of  flight  to  mission  plan.  There  are  at  least  three 
outcome  categories  to  be  considered  under  the  rubric  of  effectiveness:  (a)  mission  success,  (b)  partial  mission  success, 
and  (c)  mission  failure.  Target  kill  and  aircrew/aircraft  survival  define  the  successful  mission.  Failure  to  accomplish 
either  one  of  these  objectives  defines  failure.  Accomplishment  of  either  target  kill  or  survival  define  the  partial  success 
category. 

It  is  of  interest  from  both  a  research  and  operational  viewpoint  to  examine  the  inital  performance  levels,  the  amount 
of  improvement  that  occurs  with  limited  practice,  and  the  overall  differences  in  effectiveness  between  sets  of  plans.  The 
outcome  event  data,  arranged  according  to  first  and  last  pass  and  overall  plan  effectiveness  levels,  are  presented  in  Table 
1. 


Table  1 .  Mission  Outcome  Events 


Eveal/ 

Trial 

MImIod,  PercMt 

Target 

Destroyed 

Smfe 

E *rtm 

Target  Destroyed 
mad 

Safe  Egrc— 

On Same  Pan 

No  Target 
KID  sad 
AC  Lost 

A1 

31 

56 

25 

38 

A4 

56 

88 

56 

13 

Al-4 

47 

80 

38 

16 

B1 

44 

69 

31 

19 

B4 

56 

88 

44 

0 

Bl-4 

55 

83 

41 

5 

It  is  evident  from  examination  of  these  data  that  |a)  performance  improved  in  all  categories,  (b)  a  greater  amount 
of  improvement  took  place  during  the  first  four  passes,  (c)  the  probability  of  survival  is  far  higher  than  that  of  deslorying 
the  target,  and  (d)  terminal  performance  levels  were  similar  for  both  plans  even  though  initial  performance  levels  were 
better  on  Plan  B. 


Offensive  Effectiveness 

There  were  five  successful  target  attacks  on  the  first  pass  and  an  equal  number  of  dry  passes.  Based  on  experimenter 
observation  and  pilot  report,  four  of  the  dry  passes  were  due  to  failure  to  detect  the  target.  One  pilot  unsuccessfully 
attacked  a  threat  site  and  was  forced  out  of  the  environment  by  missile-evading  manuevering.  The  remaining  six  pilots 
delivered  ordnance  beyond  the  30-meter  kill  criterion  distance.  TTie  average  miss-distance  for  these  passes  was  129 
meters  (SD  “116  meters).  Pilots  attributed  the  misses  to  late  target  detection  and/or  defensive  maneuvering.  The  number 
of  dry  passes  dropped  to  four  on  A-2,  two  on  A-3,  and  one  on  A-4.  By  the  fourth  pass,  there  were  nine  target  kills, 
an  improvement  of  25%  from  the  first  pass.  The  average  miss-distance  for  the  no-kill  releases  was  reduced  to  99  meters 
(SD  ==  88  meters),  an  improvement  of  1 7  meters  with  less  variability. 
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By  comparison,  I hf  first  pass  of  Plan  B  was  characterized  by  only  two  dry  passes.  The  average  miss-distance  for 
the  si*  non-kill  releases  was  74  meters  (SD  =  43  meters).  By  the  fourth  pass,  there  were  no  dry  passes,  an  increase 
of  12%  in  the  number  of  kills,  and  reduction  in  the  average  non-kill  miss  distance  to  65  meters  (SD  =  36  meters).  Thus, 
as  the  amount  of  experience  in  the  scenario  increased,  the  offensive  effectiveness  increased  as  indexed,  not  only  by  the 
probability  of  a  kill,  but  by  a  reduction  in  the  number  of  dry  passes,  the  distance,  and  variability  of  the  misses. 

Table  2  presents  the  average  bomb  distances  with  associated  standard  deviations  for  all  weapons  releases  on  each 
pass.  It  is  evident  that  Plan  B  releases  were  more  accurate  and  less  variable  than  those  of  Plan  A. 


Table  2.  Average  Bomb  Scores  (Meters) 


Parameter 

A-l 

A-2 
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74 

35 

82 

.50 
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99 

22 
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76 

N 

10 

11 

14 

15 

B-l 

B-2 

B-3 

B-4 

X 

43 

26 

28 

38 

SD 

41 

26 

16 

34 

N 

13 

13 

15 

16 

Survivability 

Mission  survival  depended  on  the  pilot's  ability  to  avoid  anti-aircraft  artillery  fire,  surface-to-air  missiles,  and  the 
ground.  Probable  "frag”  damage  and  airframe  damage  due  to  excess  g-forces  were  also  recorded.  However,  the  latter 
two  events  were  not  programmed  to  have  any  consequences  for  the  pilot  or  aircraft  and  are  not  discussed  in  detad.  There 
were  no  frag  envelope  violations,  and  almost  all  passes  exceeded  9g  with  ordnance  loaded. 

As  can  be  seen  from  Table  1,  the  survival  rates  were  relatively  high.  Of  the  total  128  trails,  there  were  only  20 
(84%  survival)  in  which  the  pilot  failed  to  egress  safely.  Of  these  20  trials,  12  (60%)  losses  oceured  during  the  first 
pass  of  each  plan.  On  the  first  pass  of  Plan  A.  there  were  five  SAM  kills  and  two  terrain  crashes.  On  the  first  (>ass  of 
Plan  B.  there  were  two  SAM  kills  and  three  terrain  crashes.  Scattered  among  the  remaining  six  tnals  were  three  more 
terrain  crashes  and  nine  SAM  kills.  Altogether,  there  were  no  AAA  kills.  16  SAM  kills,  and  eight  terrain  crashes. 

AAA.  The  fact  that  there  were  no  AAA  kills  may  be  an  artifact  of  the  computer  modeling  of  the  guns  rather  than  the 
result  of  piloting  skill.  The  program  required  4  seconds  of  continuous  firing  before  an  AAA  kill  would  be  allowed.  Aircraft 
jinking  above  +4g  broke  the  firing.  A  momentary  excursion  beyond  the  4g  point  would  break  firing.  However,  a 
momentary  excursion  beyond  4g  could  be  made  (and  frequently  was)  without  significantly  altering  the  aircraft  flight  path 
(which  would  be  required  to  defeat  an  actual  threat).  A  more  acceptable  alternative  would  lie  to  use  a  model  of  gun's 
performance  capabilities  with  bullet  ballistics  modeled  directly.  However,  the  data  for  deriving  a  completely  satisfactory 
model  were  not  available  for  use  in  this  study. 

SAM.  Unlike  the  simulation  of  the  AAA.  the  SAM  performance  was  modeled  directly  in  order  to  retain  the  svstem 
as  a  “generic"  performance  model.  The  missile  was  modeled  to  approximate  the  performance  capabilities  of  a  modem, 
mobile  SAM.  except  that  it  remained  a  radar-controlled  missile  rather  than  being  converted  to  other  sensors.  The  only 
tools  the  pilot  had  to  defeat  the  missile  were  terrain  masking,  low-level  flight,  and  judicious  use  of  chaff.  A  main  limitation 
of  the  threat  modeling  in  this  scenario  was  that  only  one  missile  could  be  in  flight  at  any  given  time.  (This  restriction 
was  due  to  a  limitation  in  the  number  of  moving  models  available  on  the  ASPT).  It  was  possible,  however,  for  another 
missile  site  to  be  progressing  through  the  acquisition  to  launch  sequence  while  a  missile  was  in  flight.  The  next  missile 
could  be  fired  as  soon  as  the  previous  one  aborted. 
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An  average  of  1.4  SAMs  were  launched  per  pass  (range  0  to  5  SAMs).  There  waa  a  total  of  lb  SAM  kills  with  an 
average  92%  SAM  miaa  rate  (or  pilot  evaaiou  rate).  Of  the  SAM  taunchea  which  resulted  in  a  kill,  the  average  time  of 
flight  waa  9.9  seconds.  The  only  measured  pilot  behavior  taken  to  defeat  the  SAMa  waa  the  use  of  chaff.  (The  measurement 
system  waa  not  equipfied  to  correlate  evasive  maneuvers  with  SAM  flight.)  There  waa  a  tendency  for  the  use  of  chaff 
to  increase  as  a  function  of  triala  (1.25  bundles  per  pass  in  Plan  A  compared  with  3.6  bundles  per  pass  of  Plan  B). 
Kxporimenter  observation  and  pilot  reports  suggested  that  task  loading  or  lack  thereof  (situational  awareness)  was  a 
mediating  factor  influencing  the  pilot’s  use  of  chaff,  such  that  initial  passes  were  associated  with  heavier  workloads  and 
a  decreased  likelihood  of  chaff  dispense. 


Terrain  Avoidance 


There  were  eight  terrain  crashes,  comprising  fd/f  of  the  total  number  of  trials  and  33W-  of  the  nonsurvived  trials. 
It  is  believed  that  the  lack  of  adequate  depth  cues  in  the  computer  image  generated  visual  display  was  a  primary 
contributing  (actor  to  the  incidence  of  terrain  crashes.  However,  an  analysis  of  when  and  where  the  crashes  took  place 
suggests  that  workload  and  motivational  factors,  as  well  as  perceptual  factors,  were  involved. 

Of  the  eight  crashes,  four  (.SO'/fl  occurred  in  the  vicinity  of  the  target,  an  area  containing  artificial  vertical  visual 
cues  at  1000-foot  intervals.  These  cues,  inverted  tetrahedrons  with  black  sides  and  white  tops,  have  been  shown  to  be 
effective  height  cues  for  low-level  flight  in  noncombat-like  flight  tasks  (Martin  &  Rinalducci.  1983).  Only  one  crash 
occurred  on  ingress  (in  an  area  without  the  additional  cues).  Seven  of  the  eight  crashes  occurred  following  weapons 
release.  Four  of  these  crashes  occurred  in  the  immediate  target  vicinity  (area  with  additional  cues)  during  hard  evasive 
maniievering.  Two  occurred  after  pilots  had  successfully  maneuvered  to  a  terrain-masked,  threat-free  position;  they 
simply  flew  into  the  ground.  The  remaining  crash  occurred  as  the  pilot  was  attempting  to  break  missile  look  by  flying 
below  50  feet  (iii  an  area  with  no  additional  cues).  Thus,  it  would  appear  that  the  terrain  crashes  were  the  result  of  an 
interaction  between  the  inadequate  depth  cueing  and  the  nature  of  the  task  requirements. 


Pilot  Opinion 

Pilot  responses  to  questionnaires  and  points  of  discussion  during  debriefings  serve  a  valuable  function  in  identifying 
strengths  arid  weaknesses  in  the  basic  simulation  and  the  scenario.  They  also  helped  in  elucidating  behavioral  sequences 
that  were  not  captured  by  the  automated  performance  measurement  profiles.  The  major  topics  covered  included  scenario 
construction,  visual  display  characteristics  and  scene  content,  performance  feedback  and  measurement,  and  training 
applications. 

1.  Scenario  Constnutum.  The  most  frequent  criticism  of  the  scenario  was  the  limitation  of  a  single-ship  attack. 
It  was  felt  that  for  realism  and  effective  training  applications,  at  least  a  two-ship  attack  was  required.  Another  area  of 
concern  waa  the  relativelv  small  gaming  area.  This  pruhlem  was  compounded  by  the  use  of  mission  planning  maps  that 
were  of  a  larger  scale  than  normally  used,  leading  some  pilots  to  expect  a  larger  working  environment.  A  larger  gaming 
area  would  also  permit  the  use  of  navigation  procedures  and  instrumentation. 

2.  Visual  Duplay  and  Scene  Content.  The  most  frequently  mentioned  limitation  of  the  visual  display  was  the  lack 
of  low  altitude  depth  perception.  In  fact,  the  most  frequent  response  to  a  question  concerning  the  biggest  challenge  of 
the  scenario  was  the  absence  of  low  altitude  cues.  Difficulty  in  target  acquisition  and  the  small  gaming  area  were 
mentioned  more  frequently  than  were  AAA  and  SAM  threats.  Most  pilots  fell  that  additional  vertical  cues  would  be  more 
beneficially  placed  in  areas  of  ingress  and  egress  rather  than  in  the  immediate  target  vicinity.  The  rationale  for  this 
preference  was  that  they  felt  they  had  to  fly  an  ingress  and  egress  that  were  higher  than  planned  due  to  the  absence 
of  cues,  and  the  attack  phase  was  generally  at  a  higher  altitude  where  cues  were  not  needed  as  much.  However,  some 
pilots  reported  never  seeing  the  vertical  cues.  Without  the  cues,  pilots  reported  that  they  could  fly  in  the  100  to  500 
foot  Adi  rone  but  that  with  the  cues  they  could  fly  in  the  50  to  200  fool  AGl.  lone. 

The  visual  modeling  of  the  gun  installations  was  less  than  satisfactory  because  the  strobe  flash  feature  (used  to 
simulate  gun  firing)  continued  to  flash  even  when  the  gun  was  not  firing.  Thus,  some  pilots  were  jinking  to  evade  a  gun 
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that  was  nol  active  in  addition,  the  visibility  range  of  the  stiohe  was  »h(thtly  (greater  than  the  .'WO  foot  rangr  modeled 
for  the  gun. 


Developmental  efforts  with  this  scenario  had  identified  (he  difficulty  in  sighting  the  missiles  in  flight  as  an  item 
of  concern  to  many  of  the  combat  experienced  pilots.  For  this  study,  the  length  ol  the  missile  was  increased  to  100  feet. 
This  intentional  oversizing  was  necessary  to  overcome  the  resolution  and  luminance  levels  of  the  ASPT  visual  system. 
Pilot  opinion  regarding  the  visibility  of  the  missile  was  solicited.  There  was  considerable  disagreement  as  to  the 
requirement  to  have  such  a  missile  visible  in  flight.  Most  pilots  felt  that  some  visual  indication  of  launch,  such  as  the 
spot  of  light  used  to  indicate  bomb  impact  point,  would  be  helpful.  They  also  felt  that  a  visual  representation  of  the  missile 
signature  would  be  at  least  as  valuable  as  the  visual  acquisition  of  the  missile  itself.  The  fact  that  the  missile  was  modeled 
considerably  larger  than  reality  was  nol  seen  as  a  limitation.  In  fact,  of  the  pilots  who  observed  the  missiles  in  flight 
at  close  range,  few  were  aware  that  the  size  of  the  missile  was  soexaggerated. 

The  abstract  nature  of  the  terrain  and  target  display,  (he  low  resolution  |  approximately  bare  minutes  and  brightness 
approximately  2  foot-lamberts),  and  die  monochromatic  display  were  not  reported  as  major  limitations.  There  was  a 
general  consensus  that  more  detail  and  realism  would  be  required  for  adequate  operational  training  application. 
Difficulties  in  target  acquisition  were  generally  attributed  to  spatial,  distance  judgment  problems  stemming  from  the 
planning  maps  rather  than  from  the  characteristics  of  the  visual  display.  Additional  moving  models  for  the  display  of 
multiple  aircraft,  missiles,  and  ground  vehicles  were  v  iewed  as  a  necessary  enhancement  for  visual  displays. 

3.  Performance  Feedback  and  Vf easurentenl.  Performance  feedback  was  provided  to  the  pilot  during  flight  for  a 
SAM  or  AAA  kill  by  a  loss  of  aircraft  control  and  a  slow  spiral  to  the  ground.  A  terrain  crash  was  accompanied  hv  a 
"grey-out"  of  the  visual  display.  Bomb  impart  points  were  indicated  by  a  white  spot  on  the  ground.  If  a  threat  site  was 
destroyed,  the  visual  models  of  the  missile  were  removed  Following  ear'll  trial,  the  pilot  was  verbally  informed  of  the 
Isimb  distance  frvini  the  target  (when  appropriate)  and  of  the  source  of  a  fatal  missile  ami  was  also  given  a  summary  of 
the  threat  activity  during  the  pass.  Following  completion  of  all  eight  trials,  each  pilot  was  given  a  copy  of  the  automated 
|>erformance  measurement  printout  and  the  hand-drawn  ground  tracks.  This  tvpe  of  information  was  viewed  as  essential 
for  potential  operational  typos  of  debriefings.  The  consensus  was  that  the  most  desirable  format  would  lie  a  graphic 
printout  indicating  flight  path,  altitude,  airspeed,  g-levels.  the  {light  path  of  launched  missiles,  and  the  occurrence  of 
significant  events  such  as  weapons  release,  guns  fired,  chaff  dispensed,  and  terrain  masking.  The  remaining  numerical 
information  could  lie  displayed  at  the  bottom  of  the  page.  A  graphics  display  available  lit  the  cockpit  with  the  same 
capabilities  was  also  frequently  mentioned  as  highly  desirable.  The  main  pilot  concern  was  having  all  of  the  information 
necessary  to  reconstruct  the  pass  for  rev  iew  and  critique. 

f  Training  Applications.  The  pilots  generally  agreed  that  the'  training  (xitenlial  of  this  type  of  simulation  exercise 
was  high.  Die  greatest  lienefits  were  seen  as  the  opportunity  to  tl V  against  active  delenses  with  immediate  feedback  and 
the  opportunity  to  test  various  strategies,  (liven  a  full  mission  simulation  capability,  the  value  for  integrating  systems 
management  and  proordures  within  a  dynamic  tactical  environment  was  viewed  as  excellent.  The  potential  for  tactics 
training  was  viewed  as  high,  given  high  fidelity  models  of  the  defensive  weapons.  The  requirement  for  Ingh  fidelity 
modeling  of  threat  performance  was  considered  to  lie  the  most  important  requirement  for  operational  application.  It  was 
generallv  agreed  that  as  the  training  objectives  move  towards  a  specific  real-world  mission  rehearsal,  the  need  for  full 
aircraft  systems  and  lietter  visual  displav  would  increase. 


V.  DISCI  SSION 

The  purpose  of  a  combat-like  scenario,  such  as  the  one  investigated  in  this  study,  is  to  provide  the  pilot  an 
opportunity  to  practice  tasks  and  to  integrate  skills  that  would  la-  required  m  an  actual  mission  with  emphasis  on  those 
aspects  which  are  not  readily  practiced  in  actual  flight.  In  this  study,  the  pilot  was  given  the  opportunity  to  integrate 
fa)  target  area  mission  segment  planning  with  mission  execution,  (hi  target  area  navigation  with  target  acquisition  and 
attack,  (cl  weapons  delivery  skills  that  threat  avoidance  and  evasion,  and  (d)  defensive  maneuvering  with  low-level  flight 
and  terrain  avoidance. 
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Mission  planning  is  often  mentioned  in  fighter-oriented  publications,  such  as  Fighter  Weapons  and  Fighter  Altai  k, 
as  one  of  the  most  important,  if  not  the  most  important,  factor  contributing  to  the  ultimate  success  or  failure  ot  a  mission 
As  in  most  learning  situations,  higher  skill  levels  are  achieved  only  with  frequent  practice.  The  principles,  concepts, 
and  procedures  involved  in  tactical  planning  are  generally  taught  to  pilots  in  a  classroom  lecture  format  supplemented 
with  paper-and-pencil  planning  problems.  The  opportunity  to  plan  and  execute  a  mission  is  relatively  limited  in  frequency 
and  type  and  is  further  limited  to  a  subset  of  the  pilot  population  (e.g.,  flight  leadersl.  The  flight  simulation  environment 
could  provide  the  medium  to  plan  and  execute  a  limitless  set  of  plans  at  a  high  frequency  to  all  pilots  at  a  fraction  of 
the  cost  of  comparable  flight  experience.  Data  from  the  present  study  can  be  interpreted  as  both  supporting  the  need 
for  such  training  and  as  demonstrating  the  potential  of  the  flight  simulation  medium  to  support  the  training. 

The  fact  that  in  the  study  no  two  pilots  (from  a  group  of  experienced  fighter  pilotsl  prepared  the  same  plan  of  attack 
on  a  common  target  complex  attests  to  the  existing  variability  between  pilots.  This  variability  also  indicates  that  even 
a  simple  scenario  has  some  characteristics  of  a  good  problem  in  that  there  was  no  one  obvious  solution  and  that  a  luimtier 
of  alternatives  proved  successful.  For  an  instructional  courseware  designer,  this  In  encouraging  since  it  illustrates  the 
potential  of  using  one  problem  for  a  variety  of  instructional  purposes  without  redundancy.  The  advantages,  disadvantages, 
and  tradeoffs  among  alternative  strategies  can  be  explored.  The  use  of  one  data  base  for  multiple  purposes  is  also  a 
significant  economy  factor  ill  terms  of  computer  software  development  costs  associated  with  constructing  the  scenario 
data  base.  A  small  number  of  environments  could  be  designed  and  used  to  instruct  most  of  the  general  principles  and 
to  permit  practice  of  tactical  mission  planning. 

A  requirement  of  an  effective  data  base  and  scenario  is  that  a  tactically  valid  problem  lie  presented  and  that  tactically 
appropriate  pilot  behavior  be  facilitated.  Although  the  data  base  and  scenario  used  in  this  study  had  several  significant 
shortcomings,  they  did  provide  a  planning  problem  in  which  the  plans  made  by  majority  of  pilots  reflected  generally 
accepted  tactical  considerations:  (a)  the  need  for  careful  and  accurate  mission  planning,  (bl  heavy  reliance  on  direct 
terrain  masking,  (cl  minimized  exposure  to  ground  defense,  (d)  routes  that  minimized  the  difficulty  in  target  acquisition, 
(e)  defensive  maneuvering  airspace,  and  (f)  the  use  of  chaff  to  break  missile  lock.  The  structure  of  the  environment  made 
it  possible  to  apply  these  principles  to  a  variety  of  routes  including  both  medium-level  and  low-level  attack  profiles. 

Comparison  of  planned  versus  actual  ground  tracks  indicated  that  it  is  extremely  unlikely  that  a  pilot  will  follow 
this  aspect  of  his  plan.  Deviations  varied  in  magnitude  and  in  mission  segment  (i.e.,  ingress,  attack,  egressl.  Although 
these  changes  may  be  indicative  of  inadequate  preparation,  it  seems  more  likely  that  a  number  of  factors  contributed. 
One  which  was  clearly  operating  was  the  difficulty  of  translating  the  spatial-distance  relationship  as  depicted  on  the 
planning  maps  to  those  aspects  of  the  environment  itself.  The  skill  of  cognitive  mapping  has  been  demonstrated  to  be 
an  important  factor  affecting  mission  effectiveness,  particularly  for  unfamiliar  locations.  In  recognition  of  the  overall 
importance  of  mission  planning  and  the  difficulty  involved  for  pilot's  conceptualize  the  impact  of  their  planning  options 
in  three-dimensional  space,  the  Air  Force  is  currently  evaluating  a  mission  planning  aid:  the  Computer  Aided  Mission 
Planning  System  (CAMPS).  This  system  displays  the  planned  ground  track,  amount  of  fuel  used,  altitude,  and  other 
information.  CAMPS  has  knowledge  of  terrain  threat  location  and  capabilities  and  will  display  when  the  pilot's  plan  route 
is  vulnerable  and  to  what  degree.  The  system  also  analyzes  the  attack  plan  for  probability  of  successful  attack  (see  Si[>e. 
1981  for  more  details  on  CAMPS).  Although  CAMPS  is  still  in  a  developmental  phase,  some  automated  mission  planning 
aid  would  clearly  benefit  pilot  training. 

In  fact,  the  characteristics  of  the  map  have  been  shown  to  be  more  important  than  the  difference  between  day  and 
night  for  nap-of-the-earth  helicopter  flight  (Bynum  &  Holman,  1979).  The  use  of  o|)era(ional-type  maps  (maps  used  in 
this  study  were  not  of  the  type  or  scale  used  in  normal  operations)  may  enhance  the  realism  of  the  scenario.  However, 
it  is  doubtful  that  such  maps  would  solve  the  underlying  problems  that  the  pilot  has  in  effectively  translating  an  abstract 
two-dimensional  representation  of  the  world  into  three-dimensional  reality,  whether  it  be  simulated  or  actual  flight.  The 
Office  of  Naval  Research  has  sponsored  research  in  this  area.  TTiorndyke  and  Hayes-Roth  (1980)  have  studied  how  people 
learn  from  maps  versus  actual  navigation  and  have  discussed  a  number  of  factors  that  need  to  lie  considered.  Stasz  and 
Thomdyke  (1980)  have  studied  the  relationship  between  visual-spatial  ability  and  study  procedures  on  map  learning 
skill.  Stasz  (1980)  has  studied  the  relationship  between  visual-spatial  abilities  and  strategies  that  successful  and 
unsuccessful  individuals  exhibit.  A  simulated  scenario  such  as  the  one  used  in  this  study  could  be  used  to  extend  this 
line  of  research  into  the  dynamic  situation  and  to  study  the  relationship  between  tactical  planning  skills  and  visual-spatial 
abilities. 
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The  success  of  the  first  mission  into  an  unfamiliar  area  is  rnliia!  to  combat  readinrss.  Application  of  flight  simulator 
romliat  scenarios  to  a  training  program  will  not  eliminate  the  loss  in  effectiveness  due  to  cognitive  mapping  problems. 
A  training  research  program  addressing  this  specific  problem  needs  to  lie  undertaken,  the  results  of  which  can  lie 
incorporated  into  Ixith  flight  and  simulated  flight  combat  scenarios. 

Target  acquisition  was  not  a  major  problem  for  the  pilots  in  this  study.  The  use  of  a  single  white  building  complex, 
located  along  a  black  road,  surrounded  bv  a  gray  background  made  the  target  complex  visible  and  minimized  target 
acquisition  problems.  Problems  exfierienced  by  some  pilots  were  due  to  primarily  to  misjudgments  regarding  their 
position  relative  to  the  expected  position  of  the  target.  These  problems  were  most  common  to  plans  that  called  for  a  low 
altitude  ingress  and  pop-up  attack.  This  problem  also  occurs  in  flight.  Specific  training  in  target  acquisition  skills  could 
be  accomplished  bv  presenting  an  array  of  alternative  targets  that  vary  in  discriminahility  and  location. 

Although  the  AS! >TT  -  Ifi  does  not  have  a  fully  operational  navigation  sy  stem,  it  does  have  the  capability  of  inputting 
destination  points  for  steerage  information.  The  pilots  wanted  to  practice  using  this  system;  therefore,  most  pilots 
programmed  the  location  of  the  target  in  the  Fire  Control  Navigation  Panel  with  the  intent  of  using  the  command  steering 
displav  to  locate  the  target.  However,  the  small  size  of  the  valley  containing  the  target  made  it  an  extremely  difficult 
sequence  to  switch  from  the  navigation  display  to  the  air-to-ground  display  in  time  to  set  up  for  weapons  delivery.  In 
a  few  instances  it  was  i  lear  that  attention  paid  to  the  navigation  display  reduced  out-of-the-cockpit  visual  scanning 
sufficiently  to  make  target  acquisition  far  more  difficult  than  it  would  have  been  strictly  a  visual  search.  Thus,  even 
though  the  target  location  was  stored  for  use,  this  feature  was  rarely  used  after  the  initial  pass,  if  then.  As  a  guideline 
for  data  base  construction  for  an  advanced  fighter,  a  larger  gaming  area  should  be  planned  so  that  the  navigation  systems 
can  lie  used  effectively. 

The  rules  of  engagement  fur  this  scenario  limited  pilots  to  the  use  of  the  Continuously  Computed  Impact  Point  (CCIPl 
Iximb  release  mode.  During  their  ASPT  syllabus  training  prior  to  participating  in  this  study,  all  pilots  received  at  least 
2  hours  of  instruction  and  practice  on  all  standard  C.CIP  delivery  applications.  All  pilots  had  achieved  at  least  the 
qualification  standards  for  computed  deliveries,  and  most  pilots  could  consistently  deliver  a  Isimb  at  an  acceptable 
distance  using  a  pop-up  delivery  pattern.  Thus,  as  far  as  the  weapons  delivery  in  the  simulator  is  concerned,  the  scenario 
provided  a  transfer  situation  from  the  conventional  range  to  a  novel  environment  where  a  kill  would  be  achieved  by 
dropping  within  30  meters  of  the  target  (considerably  larger  than  their  terminal  sy  llabus  training  performance).  The  low 
kill  rate  on  the  initial  passes  may  have  been  due  to  lack  of  familiarization  with  the  environment  If  this  were  the  case, 
the  effect  should  dissipate  within  the  first  few  passes,  with  later  passes  resulting  in  a  very  high  kill  rate.  However.  Sb'T 
was  the  highest  target  kill  rate  achieved. 

Clearly,  other  factors  are  operating  to  degrade  weapons  delivery  accuracy,  even  after  eight  passes.  It  seems 
reasonable  to  assume  that  the  threat  posed  by  the  ground  defenses  was  a  large  factor  interfering  with  weapons  delivery  . 
The  capabilities  of  the  ground  of  defense  were  described  in  the  intelligence  brief,  and  the  mission  planning  activities 
forced  the  pilots  to  deal  with  the  vulnerability  of  their  plan  to  the  various  ground  defenses.  The  functional  lethality  of 
the  threat  was  obviously  affected  by  the  attack  profile  flown,  and  any  deviations  from  their  plans  introduced  uncertainty 
for  the  pilots  to  their  a  priori  analysis  of  vulnerability. 

Although  improvement  in  the  probability  of  target  destruction  across  eight  trials  was  observed  (239F  to  .Vi'S ), 
terminal  levels  in  the  50  percent  region  indicate  the  need  for  instruction  and  further  practice.  This  environment  was 
not  particularly  lethal  (no  AAA  kills,  92  percent  overall  missile  miss  rate,  and  88  percent  survivability  at  terminal 
jierformance);  the  levels  of  total  mission  effectiveness  (target  destruction  and  survival  on  the  same  pass)  were  just  slightly 
lower  than  target  kill  levels.  The  fact  that  these  performance  levels  did  not  increase  in  the  second  planning  set  indicates 
that  the  low  target  kill  levels  are  not  simply  due  to  a  poor  choice  in  the  initial  attack  plan.  The  decisions  regarding 
alternative  attack  profiles  did  not  result  in  appreciably  higher  effectiveness  levels.  Presumably  instruction  and  guided 
practice  would  elevate  performance  levels  considerably. 

Two  aspects  of  this  simulation  most  frequently  criticized  by  the  pilots  were  the  lack  of  low-level  visual  cues  and 
the  inability  to  visually  detect  a  missile  early  in  flight.  Although  the  performance  characteristics  of  the  missile  were  more 
like  those  of  the  smaller  missiles,  which  would  be  difficult  to  detect  visually  in  any  rase,  the  fact  that  they  were  modeled 
to  be  100  feet  long,  white  in  shade,  and  still  were  not  easily  detected  clearly  points  to  an  area  of  inadequate  visual  display 
quality.  Another  visual  system  inadequacy  is  reflected  in  the  difficulties  encountered  in  low-level  flight.  Four  of  the 


eight  terrain  crashes  oecured  in  an  area  devoid  of  any  ground  reference  vertical  cues.  These  crashes  were  simply  level 
flight  into  the  ground.  The  other  four  crashes  occurred  in  the  target  area  supplemented  with  artificial  vertical  cues.  These 
crashes  occurred  during  controlled  descent  (recovery  from  weapons  delivery)  and  hard  maneuvering  flight  (evasive 
jinking).  Although  these  cues  had  been  shown  to  enable  level  flight  at  slower  airspeeds,  they  evidently  do  not  provide 
sufficient  depth  and  surface  cues  for  more  extensive  maneuvering  at  high  speed. 

The  data  obtained  from  this  study  indicated  that  that  the  logic  for  the  AAA  and  chaff  features  needs  to  be  improved 
for  further  scenarios.  The  logic  for  the  AAA  was  such  that  a  kill  would  be  recorded  if  the  aircraft  was  within  the  envelope 
for  4  consecutive  seconds  at  less  than  4g.  If  the  aircraft  exceeded  4g,  the  clock  would  be  reset.  The  intent  was  to  force 
the  pilot  to  execute  evasive  maneuvers  that  would  result  in  rapid  and  significant  changes  in  the  aircraft  flight  path,  making 
it  difficult  for  a  human  to  successfully  track  the  aircraft.  However,  the  logic  which  reset  the  clock  for  any  excursion 
above  4g  (at  30  Hz  sampling  rate)  did  not  actually  require  significant  changes  in  flight  path  in  order  to  defeat  the  threat. 
Thus,  although  the  pilots  spend  an  average  of  almost  19*55:  of  the  pass  within  the  AAA  envelopes,  no  AAA  kills  were 
recorded.  Future  logics  need  to  require  sustained  maneuvering  sufficient  to  assure  tracking  difficulties. 

The  chaff  logic  denied  updated  aircraft  position  data  to  the  missile  program  for  a  time  period  ranging  from  3  to 
8  seconds  (actual  value  randomly  determined)  per  chaff  release.  During  this  time,  the  missile  travelled  ballistically  based 
on  last  aircraft  position  information.  Thus,  if  the  pilot  used  chaff  without  making  significant  changes  to  the  flight  path, 
the  chaff  would  be  ineffective.  However,  judicious  use  of  chaff  could  render  the  missile  totally  ineffective.  It  is  unlikely 
that  chaff  would  cause  such  a  strong  picture  for  such  a  long  period  of  time.  Future  logic  needs  to  take  into  account  the 
time  it  takes  a  bundle  to  separate  from  the  aircraft  and  the  time  course  of  the  dispersal  pattern. 

Debriefings  held  with  the  pilots,  using  the  performance  measurement  printouts,  made  it  clear  that  some  form  of 
graphic  representation  of  each  pass  is  a  minimum  requirement  for  effective  feedback,  instruction,  and  analysis.  A  graphic 
display  with  associated  numerical  information  of  the  type  collected  in  this  study  should  be  made  available  on  a  monitor 
located  in  the  cockpit  (as  well  as  at  an  instructor  console)  so  that  the  pilot  could  review  each  pass  prior  to  initiating 
the  next.  The  graphic  display  should  include  the  flight  path  of  each  missile  and  some  indication  of  AAA  activity.  The 
performance  measurement  system  should  be  capable  of  storing,  recalling,  replaying,  and  printing  a  hard  copy  version 
of  the  numeric  and  graphic  displays. 

According  to  pilot  opinion,  the  greatest  limitation  was  the  single  ship  scenario.  Most  pilots  felt  that  at  least  a  two-ship 
would  be  required.  Although  most  pilots  indicated  that  the  scenario  and  supporting  simulation  and  high  training  potential 
in  its  present  form,  the  overwhelming  consensus  was  that  more  tactical  training  objectives  could  be  met  with  a  multi-ship 
capability.  This  capability  would  increase  the  complexity  and  content  validity  of  the  mission  for  the  pilots.  A  multi-ship 
scenario  would  also  add  significantly  to  the  simulation  requirements.  It  could  be  accomplished  with  a  multiple  cockpit 
(and  pilot)  facility  or  with  some  form  of  computer  controlled  profile  for  the  other  members  of  the  flight.  In  order  for  a 
pilot  to  plan  a  multi-ship  mission  and  have  the  other  members  of  the  flight  displayed  and  performing  their  functions, 
novel  types  of  software  support  would  be  required.  An  alternative  would  be  to  have  a  menu  of  preprogrammed  (i.e., 
“canned”)  routines  for  the  flight  members  from  which  the  pilot  could  choose.  This  approach  would  necessarily  restrict 
the  available  options  in  the  mission  planning  phase. 

VI.  CONCLUSIONS  AND  RECOMMENDATIONS 

The  following  training  and  simulation  conclusions  emerged  from  this  study: 

1.  The  simulation  of  tactical  scenarios  provides  an  opportunity  for  the  pilot  to  integrate  skills  in  mission  planning, 
target  acquisition,  weapons  delivery,  and  threat  evasion  through  frequent  and  safe  rehearsal. 

2.  Performance  improvements  in  all  major  mission  effectiveness  categories  occur  even  without  explicit  instruction. 

3.  The  state  of  the  art  in  flight  simulation  is  adequate  to  develop  a  tactical  ground  attack  simulation  training 
program. 
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Based  on  the  objective  performance  data  and  tin-  consensus  of  pilot  opinion,  the  following  rccuiiimciidutiuii-  arc 
proposed  for  improvements  to  the  training  potential  of  tactic  a)  simulation  scenarios: 

1 .  A  multi-ship  attack  capability  should  lie  provided. 

2.  The  gaming  area  should  lx-  enlarged  to  provide  a  greater  alternative  set  of  routes  and  more  -v  stein-  oriented 
skills. 

3.  More  low  altitude  cues  are  needed  in  areas  of  likely  ingress,  egress,  and  low  atitude  threat  evasion. 

4.  Enhanced  modeling  of  ordnance  and  enemy  threat  characteristics  is  required. 

5.  Graphic  representation  of  attacks  and  threat  evasion  tactics  for  debriefing  would  lie  helpful. 

Future  research  and  development  in  this  area  should  explore  the  following:  (a)  the  instruc  tional  requirements  for 
simulated  tactical  training  in  terms  of  both  hardware  and  software,  (b)  the  proper  sequencing  of  tasks,  (cl  the  level  of 
instruction/practice  necessary  for  initial  skill  acquisition  and  medium  term  (e.g.,  6  months!  skill  retention,  (cl)  the  effects 
that  known  stressors  such  as  workload,  fatigue,  and  uncertainty  have  on  mission  performance,  and  (el  the  relationship 
between  visual-spatial  abilities  and  mission  planning  and  execution  skills.  Accumulation  of  knowledge  in  these  areas 
would  provide  the  necessary  training  technology  base  for  implementation  of  an  effective  operational  training  program. 
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